{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<style>div.container { width: 100% }</style>\n",
    "<img style=\"float:left;  vertical-align:text-bottom;\" height=\"65\" width=\"172\" src=\"../assets/PyViz_logo_wm_line.png\" />\n",
    "<div style=\"float:right; vertical-align:text-bottom;\"><h2>Tutorial 03. Customizing Visual Appearance</h2></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The previous tutorial focused on specifying elements and simple collections of them.  This one explains how the visual appearance can be adjusted to bring out the most salient aspects of your data, or just to make the style match the overall theme of your document. We'll use data in Pandas, and HoloViews, Bokeh, and Matplotlib to display the results:\n",
    "\n",
    "<div style=\"margin: 10px\">\n",
    "<a href=\"http://pandas.pydata.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:140px\" src=\"../assets/pandas.png\"/></a>\n",
    "<a href=\"http://holoviews.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"../assets/holoviews.png\"/></a>\n",
    "<a href=\"http://bokeh.pydata.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"../assets/bokeh.png\"/></a>\n",
    "<a href=\"http://matplotlib.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"../assets/matplotlib_wm.png\"/></a>\n",
    "</div>\n",
    "\n",
    "HoloViews explicitly makes the distinction between **data** and **plotting options**, which allows annotating the data with semantic metadata before deciding how to visualize the data. It also allows rendering the same object using different plotting libraries, such as Bokeh or Matplotlib.\n",
    "\n",
    "\n",
    "## Preliminaries\n",
    "\n",
    "In the [annotating your data section](./02_Annotating_Data.ipynb), ``hv.extension('bokeh')`` was used at the start to load and activate the bokeh plotting extension. In this notebook, we will also briefly use [matplotlib](www.matplotlib.org) that will be loaded, but not yet activated, by listing it second:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import holoviews as hv\n",
    "hv.extension('bokeh', 'matplotlib')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing diamond data\n",
    "\n",
    "Let's find some interesting data to generate elements from, before we consider how to customize them. Here is a dataset of information about 50,000 individual diamonds (including their weight in carats and their cut, quality, and price), which provides some rich opportunities for visualization:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diamonds = pd.read_csv('../data/diamonds.csv')\n",
    "diamonds.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One obvious thing to look at is the relationship between the mass of the diamonds given in 'carat' and their 'price'. Since the dataset is large we will sample 1,000 random datapoints from the DataFrame and plot the 'carat' column against the 'price' as a Scatter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hv.Scatter(diamonds.sample(5000), 'carat', 'price')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is clearly structure in this data, but the data is also clearly being overplotted and being squashed into a small plot. To fix the problems, we can start customizing the appearance of this object using the HoloViews [options system](http://holoviews.org/user_guide/Customizing_Plots.html). Later on in the tutorial, we will see an alternative way of avoiding overplotting using Datashader in [Working_with_Large_Datasets](./10_Working_with_Large_Datasets.ipynb). \n",
    "\n",
    "## Types of option\n",
    "\n",
    "If we want to change the appearance of what we can already see in the plot, we're no longer focusing on the data and metadata stored in the elements, but about details of the presentation.  Details specific to the final plot are stored and handled by the separate \"options\" system, not the element objects.  HoloViews allows you to set three types of options:\n",
    "\n",
    "* **plot options**: Options that tell *HoloViews* how to *construct* the plot\n",
    "* **style options**: Options that tell the underlying *plotting library* how to *style* the plot\n",
    "* **normalization options**: Options that tell *HoloViews* how to *normalize* the various elements in the plot against each other (not covered in this tutorial)\n",
    "\n",
    "\n",
    "### Plot options\n",
    "\n",
    "We noted that the data is too compressed in the x direction. Let us fix that by specifying the ``width`` plot option and additionally spread the data out along the y-axis by enabling a log axis using the ``logy`` option. We will also  enable a Bokeh 'hover' tool letting us reveal more information about each datapoint:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Scatter [width=600 logy=True tools=['hover']]\n",
    "scatter = hv.Scatter(diamonds.sample(5000), 'carat', ['price', 'cut']).redim.label(carat='Carat (ct)',\n",
    "                                                                                   price='Price ($)')\n",
    "scatter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you can see that it's still a plot of Price vs. Carat, but if you hover over a datapoint you can see that the 'cut' information is visible for each point as you visit it.\n",
    "\n",
    "The top line uses a special IPython/Jupyter syntax called the ``%%opts`` *cell magic* to specify the ``width`` plot option for all [``Scatter``](http://holoviews.org/reference/elements/bokeh/Scatter.html) objects in this cell.  ``%%opts`` accepts a simple specification where we pass the ``width=900`` keyword argument to  [``Scatter``](http://holoviews.org/reference/elements/bokeh/Curve.html) as a plot option (denoted by the *square brackets*).\n",
    "\n",
    "In this tutorial we will generally use this convenient, tab-completable IPython specific syntax, but you can of course set options in regular Python as well (see the [user guide](http://holoviews.org/user_guide/Customizing_Plots.html) for details):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scatter.options(width=500, height=200) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Scatter\n",
    "# Exercise: Try setting the height plot option of the Curve above.\n",
    "# Note: %%opts must be on the *first* line of the cell\n",
    "# Hint: the magic supports tab completion when the cursor is in the square brackets!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise: Try enabling the boolean show_grid plot option for the curve above\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Aside: ``hv.help``\n",
    "\n",
    "Tab completion helps discover what keywords are available, but you can get more complete help using the ``hv.help`` utility. For instance, to learn more about the options for ``hv.Scatter`` run ``hv.help(hv.Scatter)``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# hv.help(hv.Scatter)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Style options\n",
    "\n",
    "The plot options earlier instructed HoloViews to build a plot 600 pixels wide, when rendered with the Bokeh plotting extension. Now let's specify that the Bokeh glyph should be colored by the 'cut' column using the 'Set1' colormap and reduce the 'alpha' and 'size' of the points so we can see overlapping points better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Scatter [color_index='cut'] (alpha=0.5 cmap='Set1')\n",
    "scatter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note how the plot options applied above to ``scatter`` are remembered! The ``%%opts`` magic is used to customize the *object* displayed as output for a particular code cell: behind the scenes HoloViews has linked the specified options to the ``scatter`` object via a hidden integer id attribute.\n",
    "\n",
    "Having used the ``%%opts`` magic on ``scatter`` again, we have now associated the 'alpha', 'size' and 'cmap' *style option* to it. In the options specification syntax, style options are the keywords in *parentheses* and are keywords defined and used by Bokeh to style the corresponding [scatter glyph](http://bokeh.pydata.org/en/latest/docs/user_guide/plotting.html#scatter-markers).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise: Display scatter without any new options to verify it stays colored\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Exercise: Try setting the 'size' style options to 1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Switching to matplotlib\n",
    "\n",
    "Let us now view our curve with matplotlib using the ``%%output`` cell magic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%output backend='matplotlib'\n",
    "scatter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All our options are gone! This is because the options are associated with the corresponding plotting extension---if you switch back to 'bokeh', the options will be applicable again. In general, options have to be specific to backends; e.g. the ``size`` style option accepted by Bokeh is called ``s`` in matplotlib:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%output backend='matplotlib'\n",
    "%%opts Scatter [aspect=4 fig_size=400] (color='blue' s=4 alpha=0.2)\n",
    "scatter.select(carat=(0, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%output backend='matplotlib'\n",
    "\n",
    "# Exercise: Apply the color_index and alpha options as above, but to the matplotlib plot\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The ``%output`` line magic\n",
    "\n",
    "In the cells above we repeated ``%%output backend='matplotlib'`` to use matplotlib to render those particular cells. Instead of repeating ourselves with the cell magic, we can use a \"line magic\" (similar syntax to the cell magic but with one ``%``) to set things globally. Let us switch to matplotlib with a line magic and specify that we want SVG output instead of PNG:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%output backend='matplotlib' fig='svg'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unlike the cell magic, the line magic doesn't need to be followed by any expression and can be used anywhere in the cell. Both the ``%output`` and ``%opts`` line magics set things globally so it is recommended you declare them at the top of your notebooks to avoid confusion when re-running notebook cells. Now let us look at the SVG matplotlib output we requested:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Scatter [aspect=4 fig_size=400 xrotation=70] (color='green' s=10 marker='^')\n",
    "scatter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise: Verify for yourself that the output above is SVG and not PNG\n",
    "# You can do this by right-clicking above then selecting 'Open Image in a new Tab' (Chrome) or 'View Image' (Firefox)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Switching back to bokeh\n",
    "\n",
    "In previous releases of HoloViews, it was typical to switch to matplotlib in order to export to PNG or SVG, because Bokeh did not support these file formats. Since [Bokeh 0.12.6](https://bokeh.github.io/blog/2017/6/13/release-0-12-6/) we can now easily use HoloViews to export Bokeh plots to a PNG file, as we will now demonstrate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%output backend='bokeh'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By passing ``fig='png'`` and a ``filename='diamonds'`` to ``%output`` we can both render to PNG and save the output to file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%output fig='png' filename='diamonds'\n",
    "scatter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have requested PNG format using ``fig='png'`` and that the output should go to diamonds.png using ``filename='diamonds'``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ls *.png"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bokeh also has some SVG support, but it is not yet exposed in HoloViews."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using ``group`` and ``label``"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above examples showed how to customize by Element type, but HoloViews offers multiple additional levels of customization that should be sufficient to cover any purpose. For our last example, let us split our diamonds dataframe based on the clarity of the diamonds, selecting the lowest and highest clarity:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "low_clarity  = diamonds[diamonds.clarity=='I1']\n",
    "high_clarity = diamonds[diamonds.clarity=='IF']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now introduce the [``Spikes``](http://holoviews.org/reference/elements/bokeh/Spikes.html) element, and display it with a large width, a log y-axis and some modifications to the xticks. We can specify those options for all following [``Spikes``](http://holoviews.org/reference/elements/bokeh/Spikes.html) elements using the ``%opts`` *line magic*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%opts Spikes [width=900 logx=True xticks=8 xrotation=90] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time we will visulize the same the data as a dot graph by combining ``Scatter`` elements with the ``Spikes``, showing the distribution of prices between the low and high clarity groups. \n",
    "\n",
    "We can do this using the element ``group`` and ``label`` introduced in the [annotating your data](./02_Annotating_Data.ipynb) section as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Spikes.Diamonds.Low   (color='blue')\n",
    "%%opts Spikes.Diamonds.High  (color='red')\n",
    "hv.Spikes(  low_clarity, 'price', 'carat', group='Diamonds', label='Low') *\\\n",
    "hv.Scatter( low_clarity, 'price', 'carat', group='Diamonds', label='Low') *\\\n",
    "hv.Spikes( high_clarity, 'price', 'carat', group='Diamonds', label='High')*\\\n",
    "hv.Scatter(high_clarity, 'price', 'carat', group='Diamonds', label='High')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the color option to distinguish between the two categories of data we can now see the clear difference between the two groups, showing that diamonds with a low clarity need to have much higher mass in carats to obtain the same price. Similar techniques can be used to provide arbitrarily specific customizations when needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise: Remove the two %%opts lines above and observe the effect\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise: Give the 'Low' clarity scatter points a black 'line_color'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Optional Exercise: Try differentiating the two sets of spikes by group and not label\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Onwards\n",
    "\n",
    "We have now seen some of the ways you can customize the appearance of your visualizations. You can consult our [Customizing Plots](http://holoviews.org/user_guide/Customizing_Plots.html) user guide to learn about other approaches, including the ``hv.opts`` and ``hv.output`` utilities that let you set options per type without relying on notebook-specific syntax, and the ``.options`` method that lets you set options on particular objects. When called without any arguments, you can also use ``.opts()`` to clear any customizations that may be set on that object.\n",
    "\n",
    "You may also wish to consult the extra [A1 Exploration with Containers](./A1_Exploration_with_Containers.ipynb) tutorial, which gives examples of how the appearance of elements can be customized when viewed in containers. In the next tutorial, [Working with Tabular Data](./04_Working_with_Tabular_Data.ipynb) we will see how to use the flexibility offered by HoloViews when working with tabular data."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
